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(54) PICTURE PROCESSING APPARATUS 

(71) We, MULLARD LIMITED of Abacus House, 33 Gutter Lane, London E.C.2. , a 
British Company do hereby declare the invention, for which we pray that a patent may be 
granted to us, and the method by which it is to be performed, to be particularly descnbed m 
and by the following statement:- ^, ^ - * c 

This invention relates to picture processing apparatus capable of extractmg siMUfic^t 5 
features from a picture. More particularly the invention relates to araaratus in which the 
features are extracted by scanning the picture with an array of detectors, individual 
detectors of which remain in a fixed distribution with respect to one another during the 
scanning process. Each detector has a field of view adapted to collect radiation from an area 
of the picture of a size sufficient to allow the required detail of the features to be resoWed. 10 
TTiis area will be referred to hereinafter as the picture element. The outout of each detector 
is an analogue quantity, being a voltage or a current, related to the total amount of 
radiation being emitted by the picture element instantaneously in the field of view of the 
detector. The detector output corresponds, therefore, to the brightness of the picture 
element. The picture is essentially a planar distribution of picture elements. The planar 15 
distribution may be provided in recorded form, for example as a drawing or a photograph 
or from a scene reconstructed from a previously recorded and stored set of picture 

elements. , , . ^ . , • ^ *u_. 

Alternatively, the planar distribution may be derived, m real time, from a three 
dimensional scene using a suitable camera which images the scene into planar form. The 20 
camera may use radiation emitted from the scene in anv convenient region of the 
electromagnetic spectrum. Reflected visible and infra-red radiation and particularly 
thermal infra-red radiation may be employed but other portions of the spectrum are not 
excluded. The invention is directed to the processing of planar pi<^res hwever 
25 Patent Specification No. 1605009 (copending application 53350^3; PHB32390), which 25 
Specffication will be referred to herein as the parent Patent Specification describes and 
dak^such picture processing apparatus for application to a picture comprising a regular 
two-dimensional array of picture elements each of which is the centre picture element of a 
plurality of radiating straight lines of contiguous picture elements aligned in radial 
30 directions and in which each picture element has a brightness value, said apparatus 30 
comprising means for determining the average brightness of one or more picture elements 
adjacent to said centre picture element in such a radial direction, said average bemg taken 
- over a number of picture elements small compared to die number of picture elements m the 
maximum dunension of the picture and excluding the said centre pictore dement, and 
35 means for providing an output signal when the said average bnghtness differs from the said 35 
. centre picture element brightness by at least a predetermined threshold amount of the same 
sign on at least half of the same radial directions. 

Any given picture element may be taken as a centre picture element and is alternatively 
referred to as such hereinafter. . i ,i • >ia 

40 The invention provides a modification of the picture processing apparatus as claimed in 40 
claim 1 of the parent Patent Specification for application to a picture compnsinc regutoly 
spaced rows and regulariy spaced transverse columns of picture elements, said modified 
apparatus comprising means for comparing the brightness of a given picture element with 
(a) the brightness of a first adjacent picture element in the column oontammg said given 
45 picture element and with (b) the brightness of a second adjacent picture element m the row 45 
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containing said given picture element, and means for providing an output signal if said first 
and second picture elements both diiffer in brightness nom said given picture element by at- 
least a predetermined threshold amount of the same sign. 



20 




5 line 

picture element and being adapted to produce an output corresponi^ng to the brightness of 
the picture element in its field of view, and means for scanning said array along said 
columns of picture elements to determine the bri^tness values of said given first and 
second picture elements. 
10 The rows and colunms may be interchanged, the length of the line array lying parallel to 10 
said columns. The threshold is an amount rather than ^level. If a scanning array encounters 
a picture element which differs in brightness from the orthogonally adjacent set of picture 
elements defined above, the picture element may be a feature in the picture comprising an 
isolated point set in a darker or lighter background. The picture element may also be at the 
IS comer of a rectangular object lying ahead of the scanning array and having one side roughly 15 
parallel to the array. However, a bright or dark line or edge which is either paraOdor 
transverse to the array will generally be discriminated agaimt and the binary output not 
generated. Such a line or edge will generally have picture elements in lines either parallel to 
or transverse to the array and of more nearly equal brightness and a pair of such picture 
elements will be less likely to differ by the threshold amount. Lines and edges lying at 20 
intermediate angles to the line of the array will be less well discriminated against. Ilie 
apparatus preferentially e^ctracts point and comer features from a picture as against line and 
edj^ features. 

The output of each detector may be stored for the duration of one picture element. The 
25 previous and present outputs of each detector in the array can then be made available to the 25 
processing apparatus. The store used may be a sample-and-hold circuit or may be a delay 
line. Altematively, the picture elements may be scanned for a second time by a second 
array of detectors following behind the first array. Each detector in the second array repeats 
the scanning of a corresponding detector in the first array. The present and previous 
30 outputs are then both made available directly to the processing apparatus without the need 30 
for storage. 

The si)acing between detectors in the array may be chosen to suit the size of features 
which it is desired to detect. Likewise, the efitective spacing between present and previous 
outputs of a detector may also be chosen at will by choosmg the 'hold* period of the 
35 sample-and-hold circuit or the delay time of the delay line if storage is used or by choosing 35 
the spacing between first and second arrays. Generally, the spacing of detectors along the 
arrayis and the effective spacing between present and previous outputs would be made 
emial, since, generally most pictures have equal horizontal and vertical resolutions. 

The processmg apparatus may compare the outputs of adjacent elements in the array to 
40 determme whether their difference exceeds the threshold. But the outputs of alternate 40 
detectors may be compared so that lamer features are detected without loss of resolution 
for smaller features. In this case the effective separation of the previous outputs would be 
increased to maintain equality of horizontal and vertical resolutions. 

The processing apparatus may provide a separate final output for each detector in one 
45 row of the array considered as a centre picture element detector. Altematively a single final 45 
output ma^ be provided when a point or comer has been detected by any one or more of the 
detectors m one row of the array. The final outputs may be used to make a reconstmcted 
version of the scanned scene showing points only and comers. This reconstmcted scene may 
be viewed separately or superimposed on and in registration with the original scene by well 
50 -known optical means. * "50 

With only two detectors in the array, a feature is detected when either detector output 
exceeds its own previous output and exceeds the output of the other detector each by the 
threshold. With three or more detectors, any detector which is not at the ends of the array 
has two neighbours in the array. The feature detection logic is then that a detector output 
55 should exceed its own previous output and exceed the output of either or both of its 55 
neighbours, all by the same threshold, for a feature to be detected. 

Embodiments of the invention will now be described by way of example with reference to 
the drawings accompanying the Provisional Specification m which:- 

Figure 1 shows diagranunatically the signals produced by a line array of three detectors 
60 and their delayed outputs. 60 

Figure 2 shows a high gain differential amplifier and resistor network used in the 
processor. 

Figure 3 shows the logic functions carried out by the picture processor in block diagram 
form. 

65 Figure 4 shows a practical embodiment of the logic functions of Figure 3. 65 
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Referring to Figure 1 a line array 1 of three detectors AR, BR and CR is showni.eaoh? 
detwtor being shown schematically as a rectangle. AR. BR and CR are aito us^ 
subsequently in the description to indicate the signalou^uts of these detector^.ttelmaiff 
indicating the real-time or immediate output of the detector. The line arraj^ 1 isfyawir 
S scanning vertically upwards as indicated by the arrow 2 across a rectangular array o^RictuiB 3r 
eiemente P.E. constkuting a picture. These picture elements are shown scheInH^^caU3^a^:a^^ 
array of rectangles each equal in area to a cfetector area. In practice, the rectaimilanarrm 
of picture elements may be a continuous distribution of radiation intensity, eadiirectangpfe- 
incucating the area of the picture contributing to the single output of a det^rtor of 

10 rectangular area at an instant of sampling. In this embodiment, the outputs of each detector 10 
are stored for a time corresponding to the time taken for a detector to scan from one picmre 
element to an adjacent one. The stored outputs are therefore shown schematically as AS, 
BS and CS, each corresponding to AR, BR and CR respectively. The outputs AS, BS and 
CS are therefore the outputs AR, BR and CR respectively, delayed by^nesamphngnenod 

15 and corresponding to the previous row of picture elements scanned. This has the efiect of 15 
making available to subsequent circuitry the outputs of a block of six picture elements 

^^^^^oT^mcTs in the picture are detected by applying the following criteria to the six 
outputs, which are analogue voltage or current signals. A point or comer is said to exist at 
i20 detector AR if the signal condition is:- 20 



25 



^30 



40 



45 



50 



55 



60 



65 



-(1) 



AR - AS > T 

and AR - BR > T 

where T is a positive threshold level. ^ _ ^ , . , j 

Thus a point or comer is only said to exist at A if the signal AR exceeds its own delayed 
output AS by a threshold T and also exceeds the output BR of the adjacent detector m die 
line array by the same threshold T. Likewise for a point or comer at detector CR> 

CR - CS > T 

and CR - BR > T 



-(2) 



25 



30 



35 For a point or corner at detector BR, two possible adjacent outputs are available, AR or 35 
CR. The criteria for detector BR are therefore:- 



BR - BS > T 

and either BR - AR > T \ (3) 40 



45 



or BR - CR > T 

Thus, in applying these criteria, seven quantities must first be evaluated 

AR-AS-T (4) 

AR-BR-T ——(5) 

BR-BS-T -(6) 50 

BR-AR-T (7) 

BR-CR-T ~(8) 

CR-CS-T —(9) 

CR-BR-T — —(10) 

and an indication provided for each if it is greater than zero. Thus a binary signd is 60 
generated for each of the seven quantities, having the value '1' if the quantity is greater than 
zero or the value '0' if the quantity is zero or negative. 

Referring to Figure 2 a network of resistors and a high gam differenUal amplrfier is 
shown, one such arrangement being provided for each quantity to generate the binary 
signal. As an example, the quantity BR - AR - T is shown evaluated and the correspondmg 65 
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binary output generated. The high gain differential amplifier 3 has a single output terminal 
4 which can swing between a limited positive output and a limited negative output. 
Amplifier 3 has two input tenninals 5 and 6, being the inputs of negative and positive sense 
respectively. A positive input si^al applied to terminal 5 would appear as a gready 
amplified negative output at tenmnal 4 whereas a positive input signal to terminal 6 would 
appear as an equally amplified positive output at terminal 4. Thus, the effect is to amplify 





15 



10 terminal 6. At Terminal 5 one resistor is fed with the AR signal, the otiier Being feJwith the 10 
threshold level T. Thus half the sum of AR and T is fed to terminal 5, Hie loss of half the 
signal amplitudes at tenninals 5 and 6 is of no consequence as the gain of the amplifier 3 is 
so high. The output 4 will be driven to the limiting positive output if BR exceeds AR + T 
only very slighdy . Conversely output 4 will be ddven to the limiting negative output if AR 
+ T exceeds BR only very slightly. TTie following logic drcuits mterpret these luniting 
outputs as a '1* and an '0* respectively. 

Referring to Figure 3, the logjic function of the processor is shown schematically. In this 
processor a sinrfe output is provided if any one or more of the criteria (1), (2) and (3) above 
are satisfied. Tnus the final logic gate in the processor is a three-input *OR' gate 8 which in 
20 this embodiment inverts the si^^, providing a low voltage or *0' output if any one or more 20 
of its three inputs 9, 10 and 1 1 is a high vohage or T . The 'AND* function reqmred for each 
of the criteria (1), (2) and (3) above is provided by inverting 'AND' gates 12. 14 and 13 
reroectively. Each of these 'AND' gates has two inputs and provides a high output or *1' 
only if both inputs arc a low voltage or "O*. Consequently for quantities (4), (5) , (6), (9) and 
25 (10) above, the outputs of the corresjionding high gain amplifiers must be at a low voltage 25 
or "O* «*en the correspjonding Quantities are greater than zero. This inversion of sign when 
compared to the description of Figure 2 is achieved by reversing the input connections of 
the high gain anmlifiers concemec. For example, in amplifier 15, processing the quantity 
(4) = AR - AS - T, detector output AR and earth are applied to the resistor pair 
30 connected to the negative amplifier input, while the detector ou^ut AS and the threshold 30 
level T are applied to the resistor pair connected to the positive amplifier input. With this 
set of amplifier input connections, the output 16 of amplifier 15 will be negative when AR - 
AS - T IS positive, which has the effect of a low voltage or '0* on the input of *AND' gate 
12* 

35 For the remaining two quantities (7) and (8) required in criteria 3, an 'OR' function 35 
precedes the 'AND'^function. The *OR* gate 17 is also an inverting gate, providing a low 
voltage or '0' ou^ut if either or both of its two inputs, 18 and 19 are a high voltage or '1'. 
Consequentiy the inversion of sign provided in the other five high gain differential 
amplifiers is not needed and the input connections of Figure 2 are correct for providing the 

40 mput 18. For input 19 only the term AR is changed to CR in Figure 2. The sign of the logic 40 
used at various points in Figure 3 is indicated by bracketed 'T* 's and "0" 's at the outputs 
of amplifiers and jgates. Round bradcets give the binary value, '1' or '0', when no point or 
comer has been detected, whereas square brackets give the binary value when a point or 
comer has been detected. 

45 The output 21 of 'OR' ^te 8 is fed to a non-inverting monostable multivibrator 20. When 45 



^ ^ . . ^ _j output 22 turns transistor 23 and light-emitting 

diode 24 on. Thus, a pulse of light is emitted by diode 24 for one picture element time when 
50 a comer or point is detected by the vertical scanning of the array of detectors. An image of 50 
diode 24 is scanned in synchronism with the array, by means not shown, reconstmcting a 
single vertical line of picture information which shows only comers and points. By optical 
means not shown, this single line may be viewed superimposed upon and in registration 
with the original unprocessed picture. A distinctive mark on comers and points satisfying 
55 the threshold critena may thus be provided for an observer. 55 
Referring to Figure 4, a practical circuit embodiment of the logic of Figure 3 is shown. In 
particular, practical means are shown for generating the delayed outputs AS, BS and CS 
from the real-time outputs AR, BR and respectively. The real-tune outputs are each 
fed to sample-and-hold circuits 25, 26 and 27 respectively. Sample-and-hold circuits are 
60 known in the art and have the property of sampling a continuously applied input signjd at an 60 
instant determined by a control pulse and of storing the value of tne input present at the 
sampling instant and supplying it to an output terminal. The stored value is held and 
supplied to the output until another control pulse is applied. Then the new value of the 
sample input supplants the old stored value. Sample-and-hold (SH) circuit 25 is fed with 
65 control pulses at terminal (a) which are 10 microseconds long at intervals of 300 65 
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microseconds, corresponding to a picture element rate of 3.3 kHz. Thus, at the end of each 
.300 microsecond hold period, the output 28 of SH circuit 25 is the value AS corEegxjnding 
to the real time detector output AR then available. AR and output 28 are fed toitiiejEeffitor 
network inputs of high gain differential amplifier 29 where the Quantity AS + T -Ml is 
5 .evaluated as described with reference to Figure 3. Similarly, BR and CR .are fed SH 5 
circuits 26 and 27 respectively. The appropriate selection of the inputs AR, 1^ and £31 and 
the SH circuit outputs 30 and 31 are fed to each of the remaining six high eain«di£EGsenlial 
amplifieis 32 to 37 inclusive to evaluate the remaining six relations (5) to (10) indhisve. 31ie 
operation of the logic gates 8, 12, 13, 14 and 17 is as described with reference to Figure 3. 

10 The relative timing of the sample-and-hold circuit control pulses and the instant of 10 
sampling the output of 'OR' gate 8 are controlled by a 3.3kHz clock pulse source 38 and a 
monostable multivibrator 41. The output 42 of source 38 consists of a square wave 
generated by a multivibrator consisting of two inverting logic gates 39 and 40 connected in 
series with a feed back network composed of capacitor CI and resistors Rl and R2. The two 

15 reversals of phase in the gates 39 and 40 provide overall positive feedback giving an 15 
osdllation controlled in frequency at 3.3 kHz by the values of CI, Rl and R2. The output 42 
is a square wave of about unity mark-to-space ratio, the positive going portion of which has 
a duration of 150 microseconds. The sample-and-hold control pulse must be much less th^ 
this duration so that the storage time of the sample-and-hold circuits is as near the full 300 

20 microseconds as possible. The monostable multivibrator 41 provides such a shortened pulse 20 
in response to the positive-going portion of output 42. The stable state of the monostable 41 
is with output 43 at lo^cal T or a high positive voltage and output 44 at logical '0' or a low 
voltage. The positive gomg onset of output 42 triggers monostable 41 to produce one pulse 
of width 10 microseconds, controlled in duration by C2 and R3. Output 43 provides a single 

25 negative going pulse of this duration which turns transistors Tl, T2 and T3 on, supplying 25 
simultaneous output current pulses a, b and c respectively to the sample-and-hold cucuits 
25, 26 and 27 respectively. Output 44 simultaneously supplies a positive gomg piJse of the 
same duration to monostable 20, briefly described before with reference to Figure 3. 
Provided the output of 'OR' ^ate is a logical *0', i.e. provided a point or comer has been 

30 detected, at the time when pulse 44 arrives, monostable 20 provides a single 300 30 
microsecond duration negative going pulse on output 45, controlled m duration by C3 and 
R4, to turn ligbt-emittmg diode 24 on for one picture element time. The output 44 actually 
reaches monostable 20 before the sample-and-hold pulses a, b and c rea^ the 
sample-and-hold circuits owing to the delay in pulse propagation through transistor Tl, T2 

35 andT3. Hence the old stored values in the sample-and-hold circuits are used m the point 35 
and comer detection circuits before they are supplanted by the new stored values. These 
new values are, strictly, the values of the real-time inputs available at the end of the 10 
microsecond pulse and consequently they are actually stored for 290 microseconds. 
Referring to Figure 4, the value given to the frequency of the clock pulse source 38 of 

40 3.3kHz and the corresponding value of 300 microseconds given to the duration of the puke 40 
generated by multivibrator 20 refer to a particular example of a Figure 4 arcuit m which the 
picture element clock rate is 3.3kHz. This clock rate is determined by the rate at which 
picture elements are scanned which is settled by other parameters dependent on the 
purpose to which the picture processor is applied. These values may be adjusted to suit the 

45 application. Likewise the value of 10 microseconds given to the sample-and-hold control 45 
puke generated by multivibrator 41 would be adjusted to suit the apphc^tion. In the 
example described having a picture element rate of 3.3kHz, the values of the abeUed 
Srf and capacitors arl asfoUows. Rl = 330k, R2 = 680k, R3 = 33k R4 = lOOk^ CI = 
390pf. C2 = 390pf and C3 = 3.9nf. TTie transistors Tl, T2, T3 and T4 are aU MuUard 
- 50 BG\71. The high gain differential amplifiers 29, 32, 33, 34, 35,^^^ 50 
No. CA3060. Thelates 12, 13, 14and 17 are Motorola Type No. MC14001. TJe g^es8> 
and 40 are Type No. MC14Q25. The multivibratois 20 and 41 are Type No. MC14528. The 
supply voltage +V is 5 volts. 
WHAT WE CLAIM IS:- , . i ^♦u ♦ cc 

55 1. A modification of the picture processing apparatus as clauned m claim 1 of the parent 55 
Patent SpecificaUon for application to a picture comprising regulariy g>aced rows and 
regularly spaced transverse columns of picture elements, said modified apparatus 
comprising means for comparing the brightness of a given picture element with (a) the 
brightness of a first adjacent picture element in the column contammg said given picture 

60 element and with (b) the bnghtness of a second adjacent picture element m the row 60 
containing said given picture element, and means for providing an output signal if said first 
and second picture elements both differ in brightness firona said given picture element by at 
least a predetermined threshold amount of the same sign. . . j 

2 Picture processing apparatus as claimed in Claim 1 wherein there is provided a hne 

65 array of detectors parallel to said rows, each detector having a field of view of one picture 65 
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element and being ad^ted to produce an output corresponding to the brightness of the 
picture element in its field of view, and means for scanning said array along said columns of 
picture elements to determine the brightness valves of said given, first and second picture 
elements. 

5 3. Apparatus as claimed in Qaim 2 wherein there is provided means for storing the 5 
output of each detector for the duration of the scan of one picture element, said stored and 
direct detector outputs supplying the brightness values of said given, first and second 
picture elements. 

4. Apparatus as claimed in Qaim 2 wherein a second line array of detectors is provided 

10 parallel and adjacent to said first line array, each detector in the second array being adapted 10 
to follow and repeat the scanning of a corresponding detector in the first array, the 
simultaneously available outputs of both arrays supplying the brightness values of said 
given, first and second picture elements. 

5. Apparatus as claimed in any one of Claims 2, 3 or 4 wherein said line arrays have 



7. Apparatus substantially as described witha reference to Figures 1, 2 and 3 of tiie 
drawings accompanying the Provisional Spedfication. « 
20 8. Apparatus substantially as describes with reference to Figures 1, 2 and 4 of the 20 
drawings accompanying the Provisional Specification. 
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Apparatus as claimed in any one of Oaims 2, 3 or 4 wherein said line arrays have 
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